Skip to content

Added benchmark CRBH20 reaction energy barriers dataset#343

Open
Quantumplations wants to merge 7 commits intoddmms:mainfrom
Quantumplations:main
Open

Added benchmark CRBH20 reaction energy barriers dataset#343
Quantumplations wants to merge 7 commits intoddmms:mainfrom
Quantumplations:main

Conversation

@Quantumplations
Copy link

Pre-review checklist for PR author

PR author must check the checkboxes below when creating the PR.

Summary

  • Input data: CRBH20 (Chemical Reaction Barrier Heights 20), a set of 20 molecular geometries of reactants and their transition states (cycloreversion of heterocyclic rings).
  • Calculations: predicted reactant and transition state energies using 6 foundation models (mace-mp-0a, mace-mp-0b3, mace-mpa-0, mace-omat-0, mace-matpes-r2scan, orb-v3-consv-inf-omat, pet-mad), along with the reaction barrier height (difference between transition state and reactant energies).
  • Analysis: compare the 6 models' predicted reaction barrier heights against r2SCAN DFT reference barrier heights, metric for each model is its MAE over the 20 systems.

Linked issue

Resolves 1 issue.

Resolves #291

Progress

  • Calculations
  • Analysis
  • Application
  • Documentation

Testing

  • mace-mp-0a, mace-mp-0b3, mace-mpa-0, mace-omat-0, mace-matpes-r2scan, orb-v3-consv-inf-omat, pet-mad

New decorators/callbacks

@ElliottKasoar ElliottKasoar added the new benchmark Proposals and suggestions for new benchmarks label Feb 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

new benchmark Proposals and suggestions for new benchmarks

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Benchmarking CRBH20 (Chemical Reaction Barrier Heights) dataset against DFT

2 participants